A data buffer is a region of a physical memory storage used to temporarily store data while it is being moved from one place to another. In an application server implementing a data buffer, data objects may be stored in a buffer as they are retrieved from a client device or application before they are processed or sent elsewhere for more permanent storage. Buffers can be implemented in a fixed memory location in hardware, or by using a virtual data buffer in software, pointing at a location in the physical memory. Buffers are typically used when there is a difference between the rate at which data is received and the rate at which it can be processed, or in the case where these rates are variable. A buffer can be used to adjust timing by implementing a queue algorithm in memory, simultaneously writing data into the queue at one rate and reading it at another rate.
High-throughput, scalable, continuous events require transportation and queuing to a large data store, such as a non-relational, distributed database like Hbase.
To address this issue and provide that high-throughput, an In-Memory Buffer Service (IMBS) was recently developed as described, for example, in U.S. patent application Ser. No. 14/705,750, filed May 6, 2015, entitled “In-Memory Buffer Service,” and assigned to the assignee of the present invention, which is incorporated herein by reference in its entirety. The IMBS is a store and forward based service for objects. The IMBS is designed to provide a very low latency API to write objects to data store 130 (e.g., currently it takes 0.08 microseconds to write an object). The IMBS keeps events in memory until the number of objects reaches a configured limit and for a given time (e.g., 10 seconds). In one implementation, the IMBS can include a capture service running on an application server that receives events from a client application running on an application server to be stored in a data store. The IMBS stores the events in an in-memory bounded buffer on the application server. The in-memory bounded buffer is responsible to support a concurrently writable API, which stores events in memory and includes a buffer flush regulator that manages flushing of this buffer. The in-memory bounded buffer includes a plurality of single-threaded segments, and the capture service can write events to each segment in parallel. When a number of events stored in the in-memory bounded buffer reaches a predefined limit, the in-memory bounded buffer provides a notification to the buffer flush regulator. The in-memory bounded buffer receives a request to flush the events in the in-memory bounded buffer from a consumer executor service. The consumer executor service is responsible for consuming the events in the in-memory bounded buffer using a dynamically sized thread pool of consumer threads to read the segments of the bounded buffer in parallel to allow the events to be written directly to the data store. Events can be consumed in parallel fashion to allow for high throughput.